Safe Distillation Box

Knowledge distillation (KD) has recently emerged as a powerful strategy to transfer knowledge from pre-trained teacher model lightweight student, and demonstrated its unprecedented success over wide spectrum of applications. In spite the encouraging results, KD process \emph{per se} poses potential threat network ownership protection, since contained in can be effortlessly distilled hence exposed malicious user. this paper, we propose novel framework, termed Safe Distillation Box~(SDB), that allows us wrap virtual box for intellectual property protection. Specifically, SDB preserves inference capability wrapped all users, but precludes unauthorized users. For authorized on other hand, carries out augmentation scheme strengthen performances results student model. words, users may employ inference, only get access The proposed imposes no constraints architecture, readily serve plug-and-play solution protect network. Experiments across various datasets architectures demonstrate that, with SDB, performance an drops significantly while gets enhanced, demonstrating effectiveness SDB.